1 

An Overview of Recent Progress in the Study of 
Distributed Multi-agent Coordination 

Yongcan Cao, Member, IEEE, Wenwu Yu, Member, IEEE, 
Wei Ren, Member, IEEE, and Guanrong Chen, Fellow, IEEE 



(N. 

o 

u 

o- 

•i— > ■ 



(N 
> 

m 
cn 

o 

(N 



X 



Abstract — This article reviews some main results and progress 
in distributed multi-agent coordination, focusing on papers pub- 
lished in major control systems and robotics journals since 
2006. Distributed coordination of multiple vehicles, including 
unmanned aerial vehicles, unmanned ground vehicles and un- 
manned underwater vehicles, has been a very active research 
subject studied extensively by the systems and control community. 
The recent results in this area are categorized into several 
directions, such as consensus, formation control, optimization, 
task assignment, and estimation. After the review, a short 
discussion section is included to summarize the existing research 
and to propose several promising research directions along with 
some open problems that are deemed important for further 
investigations. 

Index Terms — Distributed coordination, formation control, sen- 
sor network, multi-agent system 



I. Introduction 

CONTROL theory and practice may date back to the 
beginning of the last century when Wright Brothers 
attempted their first test flight in 1903. Since then, control 
theory has gradually gained popularity, receiving more and 
wider attention especially during the World War II when it 
was developed and applied to fire-control systems, missile nav- 
igation and guidance, as well as various electronic automation 
devices. In the past several decades, modern control theory was 
further advanced due to the booming of aerospace technology 
based on large-scale engineering systems. 

During the rapid and sustained development of the modern 
control theory, technology for controlling a single vehicle, 
albeit higher-dimensional and complex, has become relatively 
mature and has produced many effective tools such as PID 
control, adaptive control, nonlinear control, intelligent control, 
and robust control methodologies. In the past two decades in 
particular, control of multiple vehicles has received increas- 
ing demands spurred by the fact that many benefits can be 
obtained when a single complicated vehicle is equivalently 
replaced by multiple yet simpler vehicles. In this endeavor, 
two approaches are commonly adopted for controlling multiple 
vehicles: a centralized approach and a distributed approach. 
The centralized approach is based on the assumption that a 
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central station is available and powerful enough to control 
a whole group of vehicles. Essentially, the centralized ap- 
proach is a direct extension of the traditional single-vehicle- 
based control philosophy and strategy. On the contrary, the 
distributed approach does not require a central station for 
control, at the cost of becoming far more complex in structure 
and organization. Although both approaches are considered 
practical depending on the situations and conditions of the 
real applications, the distributed approach is believed more 
promising due to many inevitable physical constraints such as 
limited resources and energy, short wireless communication 
ranges, narrow bandwidths, and large sizes of vehicles to 
manage and control. Therefore, the focus of this overview is 
placed on the distributed approach. 

In distributed control of a group of autonomous vehicles, the 
main objective typically is to have the whole group of vehicles 
working in a cooperative fashion throughout a distributed pro- 
tocol. Here, cooperative refers to a close relationship among 
all vehicles in the group where information sharing plays a 
central role. The distributed approach has many advantages in 
achieving cooperative group performances, especially with low 
operational costs, less system requirements, high robustness, 
strong adaptivity, and flexible scalability, therefore has been 
widely recognized and appreciated. 

The study of distributed control of multiple vehicles was 
perhaps first motivated by the work in distributed comput- 
ing [1], management science [2], and statistical physics [3]. 
In the control systems society, some pioneering works are 
generally referred to [4], [5], where an asynchronous agree- 
ment problem was studied for distributed decision-making 
problems. Thereafter, some consensus algorithms were studied 
under various information-flow constraints [6]-[10]. There are 
several journal special issues on the related topics published af- 
ter 2006, including the IEEE Transactions on Control Systems 
Technology (vol. 15, no. 4, 2007), Proceedings of the IEEE 
(vol. 94, no. 4, 2007), ASME Journal of Dynamic Systems, 
Measurement, and Control (vol. 129, no. 5, 2007), SIAM 
Journal of Control and Optimization (vol. 48, no.l, 2009), and 
International Journal of Robust and Nonlinear Control (vol. 
21, no. 12, 2011). In addition, there are some recent reviews 
and progress reports given in the surveys [11 ] — [15] and the 
books [16]— [23], among others. 

This article reviews some main results and recent progress 
in distributed multi-agent coordination, published in major 
control systems and robotics journals since 2006. For results 
before 2006, the readers are referred to [1 1]— [14]. 

Specifically, this article reviews the recent research results 
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in the following directions, which are not independent but 
actually may have overlapping to some extent: 

1. Consensus and the like (synchronization, rendezvous). 
Consensus refers to the group behavior that all the 
agents asymptotically reach a certain common agreement 
through a local distributed protocol, with or without 
predefined common speed and orientation. 

2. Distributed formation and the like (flocking). Distributed 
formation refers to the group behavior that all the agents 
form a pre-designed geometrical configuration through 
local interactions with or without a common reference. 

3. Distributed optimization. This refers to algorithmic devel- 
opments for the analysis and optimization of large-scale 
distributed systems. 

4. Distributed task assignment. This refers to the imple- 
mentation of a task-assignment algorithm in a distributed 
fashion based on local information. 

5. Distributed estimation and control. This refers to dis- 
tributed control design based on local estimation about 
the needed global information. 

The rest of this article is organized as follows. In Section II, 
basic notations of graph theory and stochastic matrices are 
introduced. Sections III, IV, V, VI, and VII describe the recent 
research results and progress in consensus, formation control, 
optimization, task assignment, and estimation, respectively. Fi- 
nally, the article is concluded by a short section of discussions 
with future perspectives. 

II. Preliminaries 

This section introduces basic concepts and notations of 
graph theory and stochastic matrices. 

A. Graph Theory 

For a system of n connected agents, its network topology 
can be modeled as a directed graph denoted Q = (V,W), 
where V = {v\, V2, ••• , v n } and W C V x V are, respectively, 
the set of agents and the set of edges which directionally 
connect the agents together. Specifically, the directed edge 
denoted by an ordered pair (vi,Vj) means that agent j can 
access the state information of agent i. Accordingly, agent i 
is a neighbor of agent j. A directed path is a sequence of 
directed edges in the form of {v\, U2), (i>2, 1*3), • • • , with all 
Vi e V. A directed graph has a directed spanning tree if there 
exists at least one agent that has directed paths to all other 
agents. The union of a set of directed graphs with the same 
set of agents, {Q^ , Gi m }, is a directed graph with the 
same set of agents and its set of edges is given by the union 
of the edge sets of all the directed graphs (L., j = 1, ■ ■ • ,m. 
A complete directed graph is a directed graph in which each 
pair of distinct agents is bidirectionally connected by an edge, 
thus there is a directed path from any agent to any other agent 
in the network. 

Two matrices are frequently used to represent the network 
topology: the adjacency matrix A = [ay] € K" xn with 
dij > if (vj,Vi) € W and ay = otherwise, and the 
Laplacian matrix C = [£ tJ } £ W ixn with i u = £? =1 ay 
and lij = —dij, i ^ j, which is generally asymmetric for 



directed graphs. The Laplacian C has at least one single zero 
eigenvalue with a corresponding eigenvector 1 consisting of 
all numeric 1. Here and throughout, all matrices and vectors 
are assumed to have comparable dimensions unless otherwise 
indicated. 



B. Stochastic Matrices 

A nonnegative square matrix is called (row) stochastic 
matrix if its every row is summed up to one. The product 
of two stochastic matrices is still a stochastic matrix. A row 
stochastic matrix P £ R nxn is called indecomposable and 
aperiodic if lim^oo P k = ly T for some y £ K™ [24]. 

III. Consensus 

Consider a group of n agents, each with single-integrator 
kinematics described by 

±i(t) = Ui(t), i = 1,- • • ,n, (1) 

where Xi(t) and Ui(t) are, respectively, the state and the 
control input of the zth agent. A typical consensus control 
algorithm is designed as 

n 

Ul (t) = Q ij (*) [ X 3 (*) ~ X i (*)] . ( 2 ) 

where ay(t) is the (i,j)th entry of the corresponding ad- 
jacency matrix at time t. The main idea behind (2) is that 
each agent moves towards the weighted average of the states 
of its neighbors. Given the switching network pattern due 
to the continuous motions of the dynamic agents, coupling 
coefficients ay(i) in (2), hence the graph topologies, are 
generally time-varying. It is shown in [9], [10] that consensus 
is achieved if the underlying directed graph has a directed 
spanning tree in some jointly fashion in terms of a union of 
its time-varying graph topologies. 

The idea behind consensus serves as a fundamental principle 
for the design of distributed multi-agent coordination algo- 
rithms. Therefore, investigating consensus has been a main 
research direction in the study of distributed multi-agent co- 
ordination. To bridge the gap between the study of consensus 
algorithms and many physical properties inherited in practical 
systems, it is necessary and meaningful to study consensus by 
considering many practical factors, such as actuation, control, 
communication, computation, and vehicle dynamics, which 
characterize some important features of practical systems. This 
is the main motivation to study consensus. In the following 
part of the section, an overview of the research progress in 
the study of consensus is given, regarding stochastic network 
topologies and dynamics, complex dynamical systems, delay 
effects, the sampled-data framework, asynchronous effects, 
quantization, convergence speed, and finite-time convergence, 
mainly after 2006. Several milestone results prior to 2006 can 
be found in [2], [4]-[6], [8]-[10], [25]. 
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A. Stochastic Network Topologies and Dynamics 

In multi-agent systems, the network topology among all 
vehicles plays a crucial role in determining consensus. The 
objective here is to explicitly identify necessary and/or suffi- 
cient conditions on the network topology such that consensus 
can be achieved under properly designed algorithms. 

It is often reasonable to consider the case when the network 
topology is deterministic under ideal communication chan- 
nels. Accordingly, main research on the consensus problem 
was conducted under a deterministic fixed/switching network 
topology. That is, the adjacency matrix A(t) is deterministic. 
Some other times, when considering random communication 
failures, random packet drops, and communication channel 
instabilities inherited in physical communication channels, it 
is necessary and important to study consensus problem in the 
stochastic setting where a network topology evolves according 
to some random distributions. That is, the adjacency matrix 
A{t) is stochastically evolving. 

In the deterministic setting, consensus is said to be achieved 
if all agents eventually reach agreement on a common state. 
In the stochastic setting, consensus is said to be achieved 
almost surely (respectively, in mean-square or in probability) 
if all agents reach agreement on a common state almost surely 
(respectively, in mean-square or in probability). Note that the 
problem studied in the stochastic setting is slightly different 
from that studied in the deterministic setting due to the differ- 
ent assumptions in terms of the network topology. Consensus 
over a stochastic network topology was perhaps first studied 
in [26], where some sufficient conditions on the network topol- 
ogy were given to guarantee consensus with probability one for 
systems with single-integrator kinematics (1), where the rate 
of convergence was also studied. Further results for consensus 
under a stochastic network topology were reported in [27]- 
[35], where research effort was conducted for systems with 
single-integrator kinematics [27]-[34] or double-integrator dy- 
namics [35]. Consensus for single-integrator kinematics under 
stochastic network topology has been extensively studied in 
particular, where some general conditions for almost-surely 
consensus was derived [29], [30], [33]. Loosely speaking, 
almost-surely consensus for single-integrator kinematics can 
be achieved, i.e., Xi(t) — Xj(t) — > almost surely, if and 
only if the expectation of the network topology, namely, the 
network topology associated with expectation E[A(t)], has a 
directed spanning tree. It is worth noting that the conditions 
are analogous to that in [9], [10], but in the stochastic setting. 
In view of the special structure of the closed-loop systems 
concerning consensus for single-integrator kinematics, basic 
properties of the stochastic matrices play a crucial role in 
the convergence analysis of the associated control algorithms. 
Consensus for double-integrator dynamics was studied in [35], 
where the switching network topology is assumed to be driven 
by a Bernoulli process, and it was shown that consensus can be 
achieved if the union of all the graphs has a directed spanning 
tree. Apparently, the requirement on the network topology for 
double-integrator dynamics is a special case of that for single- 
integrator kinematics due to the difference nature of the final 
states (constant final states for single-integrator kinematics and 



possible dynamic final states for double-integrator dynamics) 
caused by the substantial dynamical difference. It is still an 
open question as if some general conditions (corresponding 
to some specific algorithms) can be found for consensus with 
double-integrator dynamics. 

In addition to analyzing the conditions on the network 
topology such that consensus can be achieved, a special type of 
consensus algorithm, the so-called gossip algorithm [36], [37], 
has been used to achieve consensus in the stochastic setting. 
The gossip algorithm can always guarantee consensus almost 
surely if the available pairwise communication channels satisfy 
certain conditions (such as a connected graph or a graph 
with a directed spanning tree). The way of network topology 
switching does not play any role in the consideration of 
consensus. 

The current study on consensus over stochastic network 
topologies has shown some interesting results regarding: (1) 
consensus algorithm design for various multi-agent systems, 

(2) conditions of the network topologies on consensus, and 

(3) effects of the stochastic network topologies on the con- 
vergence rate. Future research on this topic includes, but not 
limited to, the following two directions: (1) when the network 
topology itself is stochastic, how to determine the probability 
of reaching consensus almost surely? (2) compared with the 
deterministic network topology, what are the advantages and 
disadvantages of the stochastic network topology, regarding 
such as robustness and convergence rate? 

As is well known, disturbances and uncertainties often exist 
in networked systems, for example, channel noise, commu- 
nication noise, uncertainties in network parameters, etc. In 
addition to the stochastic network topologies discussed above, 
the effect of stochastic disturbances [38], [39] and uncertain- 
ties [40] on the consensus problem also needs investigation. 
Study has been mainly devoted to analyzing the performance 
of consensus algorithms subject to disturbances and to present- 
ing conditions on the uncertainties such that consensus can be 
achieved. In addition, another interesting direction in dealing 
with disturbances and uncertainties is to design distributed 
local filtering algorithms so as to save energy and improve 
computational efficiency. Distributed local filtering algorithms 
play an important role and are more effective than traditional 
centralized filtering algorithms for multi-agent systems. For 
example, in [41]-[43] some distributed Kalman filters are 
designed to implement data fusion. In [44], by analyzing 
consensus and pinning control in synchronization of complex 
networks, distributed consensus filtering in sensor networks is 
addressed. Recently, Kalman filtering over a packet-dropping 
network is designed through a probabilistic approach [45]. 
Today, it remains a challenging problem to incorporate both 
dynamics of consensus and probabilistic (Kalman) filtering 
into a unified framework. 

B. Complex Dynamical Systems 

Since consensus is concerned with the behavior of a group 
of vehicles, it is natural to consider the system dynamics 
for practical vehicles in the study of the consensus problem. 
Although the study of consensus under various system dynam- 
ics is due to the existence of complex dynamics in practical 
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systems, it is also interesting to observe that system dynamics 
play an important role in determining the final consensus 
state. For instance, the well-studied consensus of multi-agent 
systems with single-integrator kinematics often converges to a 
constant final value instead. However, consensus for double- 
integrator dynamics might admit a dynamic final value (i.e., a 
time function). These important issues motivate the study of 
consensus under various system dynamics. 

As a direct extension of the study of the consensus prob- 
lem for systems with simple dynamics, for example, with 
single-integrator kinematics or double-integrator dynamics, 
consensus with general linear dynamics was also studied 
recently [46]-[51], where research is mainly devoted to finding 
feedback control laws such that consensus (in terms of the 
output states) can be achieved for general linear systems 

±i = Axi + Bu^ y l = Cx u (3) 

where A, B, and C are constant matrices with compatible 
sizes. Apparently, the well-studied single-integrator kinematics 
and double-integrator dynamics are special cases of (3) for 
properly choosing A, B, and C. 

As a further extension, consensus for complex systems has 
also been extensively studied. Here, the term consensus for 
complex systems is used for the study of consensus problem 
when the system dynamics are nonlinear [52]-[85] or with 
nonlinear consensus algorithms [86]-[88]. Examples of the 
nonlinear system dynamics studied in the consensus problem 
include: 

• Nonlinear oscillators [57]. The dynamics are often as- 
sumed to be governed by the Kuramoto equation 




sw(6j-6i), 1 = 1,2,... ,N, (4) 



j'=i 



where 6{ and u>i are, respectively, the phase and natural 
frequency of the ith oscillator, N is the number of 
oscillators, and K is the control gain. Generally, the 
control gain K plays a crucial role in determining the 
synchronizability of the network. 
. Complex networks [55], [67]-[71], [80]-[82], [84], [89]. 
The dynamics are typically represented as 

N 

Xi(t) =f( Xi (t)) + c J2 aij(t)?(xj(t) ~ a*(t)), 

i = l,2,... ,N, (5) 

where X{ = (xn,Xi2, ■ ■ ■ , Xi n ) T £ R™ is the state vector 
of the ith node, / : M" R" is a nonlinear vector func- 
tion, c is the overall coupling strength, A(t) = [a.Lj(t)] is 
the outer coupling matrix with Oy(t) = 1 if node i and 
node j are connected at time t but otherwise oy (t) = 0, 
with a,ii(t) = ki (degree of node i), and T is a general 
inner coupling matrix describing the inner interactions 
between different state components of agents. It is easy 
to see that model (1) with control input (2) is a special 
case of (5) with / = 0. 



. Nonholonomic mobile robots [54], [77], [83], [90]. The 
dynamics are described by 

Xi = UiCos^,^ = UiSm6i,6i =u>i,i = 1,- • ,N, 

(6) 

where [xi,yi] denotes the location of the ith agent, 
and Ui and u>i denote, respectively, its translational and 
rotational velocity. Note that there are three states and two 
control inputs. Therefore, the dynamics for nonholonomic 
mobile robots are underactuated. This poses substantial 
difficulties in designing proper consensus algorithms with 
corresponding stability analysis. 
. Rigid bodies and the like [63]-[66], [78], [79], [85]. One 
typical (but not unique) description of the dynamics is 

M i (q i )q i +Ci{q l ,qi)q i +gi{q i ) = T i} i = 1,- • ,N, (7) 

where qi £ M. p is the vector of generalized coordinates, 
Mi(qi) £ W xp is the symmetric positive-definite inertia 
matrix, Ci(q-i,qi)qi £ R p is the vector of Coriolis and 
centrifugal torques, gi(qi) is the vector of gravitational 
torques, and Tj £ MP is the vector of torques produced 
by the actuators associated with the ith agent. In practice, 
the dynamics of many mechanical systems are similar 
to (7). A notable property regarding the dynamics of rigid 
bodies is that Mi(qi)—2Ci{qi,qi) is skew-symmetric {i.e., 
z T [Mi(qi) ~ 2C l (q l ,q l )]z = for all z £ W), which 
plays a crucial role in finding Lyapunov functions and 
the subsequent stability analysis. 
Although the aforementioned system dynamics are different 
from the well-studied single-integrator kinematics and double- 
integrator dynamics, the main research problem is same, 
namely, to drive all agents to some common states through 
local interactions among agents. Similarly to the consensus 
algorithms proposed for systems with simple dynamics, the 
consensus algorithms used for these complex models are also 
based on a weighted average of the state differences, with 
some additional terms if necessary. Main research work has 
been conducted to design proper control algorithms and derive 
necessary and/or sufficient conditions such that consensus can 
be achieved ultimately. 

Note that although the objective is same, i.e., to guarantee 
reaching agreement on some final states, the problem is more 
complicated due to the nonlinearity of the closed-loop systems. 
In addition, most properties of stochastic matrices cannot 
be directly applied to their convergence analysis. The main 
techniques used in their stability analysis include dissipativity 
theory [52], nonsmooth analysis [87], [88], [90], and especially 
Lyapunov functions [57], [63], [65], [66], [90], [91]. 

One particular interesting topic is synchronization in com- 
plex networks which has been widely investigated in the 
past decade [92], [93]. Mathematically, the definitions for 
synchronization in complex networks and consensus in multi- 
agent systems are very similar, so to differentiate these two 
definitions and promote research exchanges in these two 
topics, their differences are briefly summarized below. 

1) Different Asymptotic States (Nonlinear Dynamics ver- 
sus Linear Dynamics). In the studies of synchronization in 
complex networks, researchers focus on synchronization with 



5 



self-nonlinear dynamics where each single system is unstable 
and thus the final asymptotic synchronization state is typically 
time-varying [92], [94]. However, in the investigations of 
multi-agent systems, the individual self-dynamics on each 
system is usually linear or zero and therefore the asymptotic 
consensus state is usually a constant [6], [8]. 

2) Different Focuses (Known Connectivity versus Time- 
varying Distributed Protocol). In synchronization of complex 
networks, the aim is to reveal how the network structure, which 
is known in priori, affects the nonlinear collective dynamics 
[92], [93], while the aim of consensus in multi-agent systems 
is to figure out how the designed distributed local protocol 
concerning mobile time-varying network structure affects the 
consensus behavior [6], [8], [9]. 

3) Different Approaches (Lyapunov Method versus Stochas- 
tic Matrix Theory). Since both complex networks and multi- 
agent systems are networked systems, algebraic graph theory 
[95] is a common approach to use. Because of the nonlinear 
terms in synchronization of complex networks, Lyapunov 
function method is usually used together with matrix theory 
[55], [89], [94]. In order to show consensus in multi-agent 
systems with time-varying network structures, stochastic ma- 
trix theory [4]-[6], [9] and convexity analysis [10] are often 
applied. 

4) Different Inner Matrices T (General Inner Matrix versus 
Particular Inner Matrix). In the typical simple consensus 
model, the inner matrices V are usually an identity matrix and 

a rank-one matrix ( J J for multi-agent systems with 

single-integrator kinematics [8] and double-integrator dynam- 
ics [96]-[98], respectively. In consensus models with higher- 
order dynamics [99], the inner matrix is similar. However, the 
inner matrix in system (5) is a general one. 

In summary, synchronization in complex networks focuses 
on nonlinear dynamics while consensus in multi-agent systems 
focuses on distributed cooperative control, and thus different 
approaches are utilized. 

The current research on consensus with complex systems 
focuses on fully-actuated systems although consensus for non- 
holonomic mobile robots [90], which are typical underactuated 
systems. Note that many mechanical devices are described 
by systems with underactuation. Therefore, it is important to 
develop appropriate consensus algorithms for underactuated 
systems. 

C. Delay Effects 

Time delay appears in almost all practical systems due 
to several reasons: (1) limited communication speed when 
information transmission exists; (2) extra time required by 
the sensor to get the measurement information; (3) compu- 
tation time required for generating the control inputs; and 
(4) execution time required for the inputs being acted. In 
general, time delay reflects an important property inherited 
in practical systems due to actuation, control, communication, 
and computation. 

Knowing that time delay might degrade the system perfor- 
mance or even destroy the system stability, studies have been 
conducted to investigate its effect on system performance and 



stability. A well-studied consensus algorithm for (1) is given 
in (2), where it is now assumed that time delay exists. Two 
types of time delays, communication delay and input delay, 
have been considered in the literature. Communication delay 
accounts for the time for transmitting information from origin 
to destination. More precisely, if it takes time for agent i 
to receive information from agent j, the closed-loop system 
of (1) using (2) under a fixed network topology becomes 

n 

±i (t) = <Hi (*) [xj (t - Tij ) - Xi(t)} . (8) 

3=1 

An interpretation of (8) is that at time t, agent i receives 
information from agent j and uses data Xj(t — T,j) instead of 
Xj (t) due to the time delay. Note that agent i can get its own 
information instantly, therefore, input delay can be considered 
as the summation of computation time and execution time. 
More precisely, if the input delay for agent i is given by Tf, 
then the closed-loop system of (1) using (2) becomes 

ii(t) = - Tf) -xi(t- If)]. (9) 

Clearly, (8) refers to the case when only communication 
delay is considered while (9) refers to the case when only 
input delay is considered. It should be emphasized that both 
communication delay and input delay might be time-varying 
and they might co-exist at the same time. 

In addition to time delay, it is also important to consider 
packet drops in exchanging state information. Fortunately, 
consensus with packet drops can be considered as a special 
case of consensus with time delay, because re-sending packets 
after they were dropped can be easily done but just having time 
delay in the data transmission channels. 

Thus, the main problem involved in consensus with time 
delay is to study the effects of time delay on the convergence 
and performance of consensus, referred to as consensusabil- 
ity [100]. 

Because time delay might affect the system stability, it is 
important to study under what conditions consensus can still 
be guaranteed even if time delay exists. In other words, can 
one find conditions on the time delay such that consensus can 
be achieved? For this purpose, the effect of time delay on the 
consensusability of (1) using (2) was investigated. When there 
exists only (constant) input delay, a sufficient condition on the 
time delay to guarantee consensus under a fixed undirected 
interaction graph is presented in [8]. Specifically, an upper 
bound for the time delay is derived under which consensus 
can be achieved. This is a well-expected result because time 
delay normally degrades the system performance gradually but 
will not destroy the system stability unless the time delay is 
above a certain threshold. Further studies can be found in, 
e.g., [101] — [1 11], which demonstrate that for (1) using (2), 
the communication delay does not affect the consensusability 
but the input delay does. In a similar manner, consensus with 
time delay was studied for systems with different dynamics, 
where the dynamics (1) are replaced by other more complex 
ones, such as double-integrator dynamics [97], [1 12]— [1 19], 
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complex networks [120]— [123], rigid bodies [124], [125], and 
general nonlinear dynamics [126]. 

In summary, the existing study of consensus with time delay 
mainly focuses on analyzing the stability of consensus algo- 
rithms with time delay for various types of system dynamics, 
including linear and nonlinear dynamics. Generally speaking, 
consensus with time delay for systems with nonlinear dy- 
namics is more challenging. For most consensus algorithms 
with time delays, the main research question is to determine 
an upper bound of the time delay under which time delay 
does not affect the consensusability. For communication delay, 
it is possible to achieve consensus under a relatively large 
time delay threshold. A notable phenomenon in this case is 
that the final consensus state is constant. Considering both 
linear and nonlinear system dynamics in consensus, the main 
tools for stability analysis of the closed-loop systems include 
matrix theory [102], [103], Lyapunov functions [120], [121], 
frequency-domain approach [106], passivity [122], and the 
contraction principle [101]. 

Although consensus with time delay has been studied 
extensively, it is often assumed that time delay is either 
constant or random. However, time delay itself might obey its 
own dynamics, which possibly depend on the communication 
distance, total computation load and computation capability, 
etc. Therefore, it is more suitable to represent the time delay 
as another system variable to be considered in the study of the 
consensus problem. In addition, it is also important to consider 
time delay and other physical constraints simultaneously in the 
study of the consensus problem. 

D. Sampled-data Framework 

The previous three subsections describe the main research 
work in the study of the consensus problem. The following 
introduces a few other aspects, namely, sampled-data frame- 
work, quantization, asynchronous effect, convergence speed, 
and finite-time convergence, that have been considered in the 
consensus problem as well. Among these topics, sampled- 
data framework, quantization, and asynchronous effects are 
considered due to some physical limitations in practical sys- 
tems while convergence speed and finite-time convergence are 
concerned with the performance for some proposed consensus 
algorithms. 

Due to the limitations in the measurement and control units, 
it is often impossible to acquire information measurements 
at an arbitrarily fast speed and to execute the control in- 
puts instantaneously. Accordingly, the closed-loop systems are 
modeled in a hybrid fashion. That is, the system plant is 
described in a continuous-time setting while the measurements 
and control inputs are described in a piecewise constant 
fashion. For instance, in a sampled-data setting, (2) becomes 

n 

= Ui(fcT) =J2aij{kT)[ Xj {kT) - x r {kT)] (10) 

for kT < t < (k + 1)T, where T is the sampling period 
and k is the discrete-time index. Essentially, (10) is a zero- 
order-hold version of Ui(t) in the sense that the control inputs 



remain unchanged during each sampling period. Under this cir- 
cumstance, consensus is studied in a sampled-data framework, 
called sampled-data consensus, which reflects the limitations 
inherited in physical measurement and control units. Mean- 
while, it is also important to point out that the sampled-data 
consensus algorithms require much less information exchange 
and computational power than the continuous-time consensus 
algorithms. Accordingly, consensus under the sampled-data 
framework deserves certain consideration. 

Sampled-data consensus was investigated in, e.g., [115], 
[127]— [136]. Consensus for systems with single-integrator 
kinematics (1) was studied under a sampled-data framework 
with a fixed or a switching network topology, in [128], 
[129], where some necessary and/or sufficient conditions were 
presented to guarantee achieving consensus. Sampled-data 
consensus of systems with double-integrator kinematics was 
studied under fixed or switching network topologies in [115], 
[127], [130]-[136]. Due to the fact that an infinitely large 
sampling period will cause no information exchange among 
the agents, the main research question is to find conditions 
on the sampling period T, which might be time-varying, 
such that consensus can be achieved. The conditions on the 
network topology for the sampled-data closed-loop systems 
are mostly similar to that for the continuous-time closed-loop 
systems. Note that the existing research on consensus in a 
sampled-data framework mainly focuses on the simple system 
dynamics and thus the closed-loop system can be represented 
in terms of a linear matrix equation. The corresponding net- 
work stability can be analyzed by investigating the properties 
of the system matrices constructed based on the proposed 
consensus algorithms and the given network topology. Various 
approaches, including Lyapunov functions [127], [131], matrix 
theory [115], [132], [135], [136], stochastic matrices [133], 
and linear matrix inequalities [130], [131], have been adopted, 
and some necessary and/or sufficient conditions have been 
derived for guaranteeing sampled-data consensus. 

It is natural to consider the sampled-data effect for consen- 
sus with general linear or nonlinear dynamics (see, e.g., [137]). 
In addition, it is meaningful to consider the case when all 
vehicles do not necessarily share the same sampling period or 
the sampling period is not necessarily constant. Accordingly, 
it is expected that a careful design of the sampling periods 
(associated with the proposed algorithms) might lead to the 
optimization of the closed-loop systems under the proposed 
algorithms subject to certain cost functions, such as maximum 
convergence rate and minimum total information exchange. In 
another word, it is intriguing to move from analysis to design 
when investigating the consensus problem in a sampled-data 
framework. 

E. Asynchronous Effects 

In most existing research of the consensus problem, it is 
assumed that all agents update their states synchronously, 
which requires a synchronized clock for the whole group of 
agents. However, such a synchronized clock might not exist 
in real applications. This motivates the design of consensus 
algorithms in an asynchronous fashion; that is, each agent 
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updates its own states regardless of the update times of other 
agents. 

In most studies of the asynchronous consensus problem 
for networked systems, due to the intrinsic technical dif- 
ficulties, usually only single-integrator kinematics (1) and 
double-integrator dynamics are considered. In [103], such 
an asynchronous consensus problem with time delay was 
investigated by utilizing some basic properties of stochastic 
matrices. Similarly in [138], the asynchronous consensus prob- 
lem was studied by using matrix theory and graph theory, and 
in [139], by employing the paracontracting theory. In [140], 
the authors studied the asynchronous consensus problem for 
double-integrator dynamics and presented sufficient conditions 
to guarantee consensus, where a condition based on linear 
matrix inequalities was given. 

Note that consensus in an asynchronous fashion has been 
considered mainly for single-integrator kinematics and double- 
integrator dynamics but not for other system dynamics. For 
certain linear systems, it might be expected that asynchronous 
communication does not affect the consensusability as shown 
in [103], [138] for single-integrator kinematics. However, a 
similar conclusion may not hold for systems with general 
dynamics, especially nonlinear dynamics. It is important to 
quantify the effects of the asynchronous communication on 
the consensus problem. 

F. Quantization 

Quantized consensus has been studied recently with motiva- 
tion from digital signal processing. Here, quantized consensus 
refers to consensus when the measurements are digital rather 
than analog therefore the information received by each agent 
is not continuous and might have been truncated due to digital 
finite precision constraints. Roughly speaking, for an analog 
signal s, a typical quantizer with an accuracy parameter 5, 
also referred to as quantization step size, is described by 
Q(s) = q(s,S), where Q(s) is the quantized signal and q(-, •) 
is the associated quantization function. For instance [141], a 
quantizer rounding a signal s to its nearest integer can be 
expressed as Q(s) = n, if s € [(n— 1/2)5, (n+l/2)5], nEZ, 
where Z denotes the integer set. Note that the types of 
quantizers might be different for different systems, hence Q(s) 
may differ for different systems. Due to the truncation of 
the signals received, consensus is now considered achieved if 
the maximal state difference is not larger than the accuracy 
level associated with the whole system. A notable feature 
for consensus with quantization is that the time to reach 
consensus is usually finite. That is, it often takes a finite 
period of time for all agents' states to converge to an accuracy 
interval. Accordingly, the main research is to investigate the 
convergence time associated with the proposed consensus 
algorithm. 

Quantized consensus was probably first studied in [141], 
where a quantized gossip algorithm was proposed and its 
convergence was analyzed. In particular, the bound of the 
convergence time for a complete graph was shown to be poly- 
nomial in the network size. In [142], coding/decoding strate- 
gies were introduced to the quantized consensus algorithms, 



where it was shown that the convergence rate depends on 
the accuracy of the quantization but not the coding/decoding 
schemes. In [143], [144], quantized consensus was studied via 
the gossip algorithm, with both lower and upper bounds of the 
convergence time derived in terms of the network size. Further 
results regarding quantized consensus were reported in [145]- 
[152], where the main research was also on the convergence 
time for various proposed quantized consensus algorithms as 
well as the quantization effects on the convergence time. It is 
intuitively reasonable that the convergence time depends on 
both the quantization level and the network topology. It is 
then natural to ask if and how the quantization methods affect 
the convergence time. This is an important measure of the 
robustness of a quantized consensus algorithm (with respect 
to the quantization method). 

Note that it is interesting but also more challenging to study 
consensus for general linear/nonlinear systems with quantiza- 
tion. Because the difference between the truncated signal and 
the original signal is bounded, consensus with quantization 
can be considered as a special case of one without quantization 
when there exist bounded disturbances. Therefore, if consensus 
can be achieved for a group of vehicles in the absence of 
quantization, it might be intuitively correct to say that the 
differences among the states of all vehicles will be bounded 
if the quantization precision is small enough. However, it is 
still an open question to rigorously describe the quantization 
effects on consensus with general linear/nonlinear systems. 

G. Convergence Speed 

In addition to the study on the consensus problem with 
physical constraints mentioned in the previous subsections, 
it is also important to study the control performance of the 
consensus problem. From the control's perspective, it is natural 
to propose proper control algorithms and analyze the stabil- 
ity, and to optimize the proposed control algorithms under 
certain control performance indexes. In this subsection, the 
convergence speed problem is reviewed, which is an important 
performance measure for consensus algorithms. 

For (1) using algorithm (2) in a connected undirected graph, 
the worst-case convergence speed was shown in [8] to be the 
Laplacian spectral gap min x ^o,i r A-=o X \\x{* = wnere 
is an all-zero column vector, X = [x\, ■ ■ ■ , x n ] T , C is the 
Laplacian matrix with A2 being smallest nonzero eigenvalue. 
Here, one should recall that the smallest eigenvalue of a 
Laplacian matrix for a connected undirected graph is zero and 
all the other eigenvalues are positive. 

In order to increase the convergence speed, the above 
spectral gap should be enlarged. For this purpose, an iterative 
algorithm was proposed in [153] to maximize the above 
spectral gap, by employing a semidefinite programming solver. 

Other than the smallest nonzero eigenvalue of the Laplacian 
matrix, another commonly used measure for the convergence 
speed is the following ratio, introduced in [154], [155]: 

(||X(£) — X*\\ \ 
ll*(o)-*iJ ' (U) 

where X* represents the final equilibrium given by <rl, where 
a is a constant. 
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In [154], this problem of finding the fastest convergence 
speed was casted into a semidefinite programming problem. 
Furthermore, the convergence speed defined by (11) was 
studied in both deterministic and stochastic settings. In the 
deterministic setting, it was studied in [155]— [157] with esti- 
mation of lower bounds. In the stochastic setting, this problem 
was studied in [26], [29], [158], with a per-step convergence 
factor introduced and discussed in [158], which itself can be 
considered a measure of the convergence speed. 

The existing study mainly focuses on the analysis of the 
convergence speed under various network topologies and op- 
timization of the convergence speed for certain given network 
topologies. Considering the fact that consensus under different 
network topologies may demonstrate different convergence 
speeds, a natural question is how to design an optimal (switch- 
ing) network topology with proper adjacency matrix such that 
optimal convergence speed can be achieved. 



/. Remarks 

In summary, the existing research on the consensus problem 
has covered a number of physical properties for practical 
systems and control performance analysis. However, the study 
of the consensus problem covering multiple physical properties 
and/or control performance analysis has been largely ignored. 
In other words, two or more problems discussed in the above 
subsections might need to be taken into consideration simul- 
taneously when studying the consensus problem. In addition, 
consensus algorithms normally guarantee the agreement of 
a team of agents on some common states without taking 
group formation into consideration. To reflect many practical 
applications where a group of agents are normally required 
to form some preferred geometric structure, it is desirable to 
consider a task-oriented formation control problem for a group 
of mobile agents, which motivates the study of formation 
control presented in the next section. 



H. Finite-tune Convergence 

As an extension of the study of convergence speed for the 
consensus problem, finite-time consensus, reaching consensus 
in a finite time, has also been studied recently. Compared 
with most existing research on the consensus problem, finite- 
time consensus demonstrates a disturbance rejection property 
and robustness against uncertainties. In addition, due to the 
finite-time convergence, it is often possible to decouple the 
consensus problem from other control objectives when they 
are considered simultaneously. 

For a group of n agents, e.g. with dynamics (1), the 
objective is to design Ui(t) such that Xi(t) = Xj(t) for t>T, 
where T is a constant. Here, T is called the consensus time. 

Finite-time consensus for networked systems with single- 
integrator kinematics (1) in the continuous-time setting was 
solved in [86], [88], [159]— [163]. Finite-time consensus for 
networked systems with double-integrator dynamics in the 
continuous-time setting was studied in [164]. An important 
common characteristic of the proposed algorithms is the use 
of the signum function. It is well known that linear consensus 
algorithms can normally guarantee asymptotic convergence, 
but not finite-time convergence. On the contrary, the finite-time 
consensus algorithms developed in [86], [88], [159]— [161], 
[163], [164], which utilize the signum function, are able to 
do so. 

Note that the existing research on finite-time consensus 
mainly focuses on systems with simple dynamics, such as 
single-integrator kinematics and double-integrator dynamics, 
in the continuous-time setting. Because many practical sys- 
tems are better and more proper to be described by general 
linear/nonlinear dynamics, it is natural to study finite-time 
consensus for systems with general linear/nonlinear dynamics 
in the future. In addition, it is interesting to study finite-time 
consensus in the discrete-time setting. Some recent work on 
this topic can be found in [165], [166], where the objective is 
to compute the final consensus value for all agents in a finite 
number of steps. 



IV. Formation Control 

Compared with the consensus problem where the final 
states of all agents typically reach a singleton, the final states 
of all agents can be more diversified under the formation 
control scenario. Indeed, formation control is more desirable 
in many practical applications such as formation flying, co- 
operative transportation, sensor networks, as well as combat 
intelligence, surveillance, and reconnaissance. In addition, the 
performance of a team of agents working cooperatively often 
exceeds the simple integration of the performances of all 
individual agents. For its broad applications and advantages, 
formation control has been a very active research subject in 
the control systems community, where a certain geometric 
pattern is aimed to form with or without a group reference. 
More precisely, the main objective of formation control is 
to coordinate a group of agents such that they can achieve 
some desired formation so that some tasks can be finished by 
the collaboration of the agents. Generally speaking, formation 
control can be categorized according to the group reference. 
Formation control without a group reference, called formation 
producing, refers to the algorithm design for a group of agents 
to reach some pre-desired geometric pattern in the absence 
of a group reference, which can also be considered as the 
control objective. Formation control with a group reference, 
called formation tracking, refers to the same task but following 
the predesignated group reference. Due to the existence of 
the group reference, formation tracking is usually much more 
challenging than formation producing and control algorithms 
for the latter might not be useful for the former. As of today, 
there are still many open questions in solving the formation 
tracking problem. 

The following part of the section reviews and discusses 
recent research results and progress in formation control, 
including formation producing, formation tracking, and con- 
nectivity maintenance for consensus and formation control, 
mainly accomplished after 2006. Several milestone results 
prior to 2006 can be found in [167]-[169]. 
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A. Formation Producing 

The existing work in formation producing aims at analyzing 
the formation behavior under certain control laws, along with 
stability analysis. 

1) Matrix Theory Approach: Due to the nature of multi- 
agent systems, matrix theory has been frequently used in the 
stability analysis of their distributed coordination. 

Note that consensus input to each agent (see e.g., (2)) is 
essentially a weighted average of the differences between the 
states of the agent's neighbors and its own. As an extension 
of the consensus algorithms, some coupling matrices were 
introduced here to offset the corresponding control inputs by 
some angles [170]— [173]. For example, given (1), the control 
input (2) is revised as 

n 

u<(t) =^2a ij {t)C[x j (t) - Xi{t)], (12) 

where C is a coupling matrix with compatible size. If Xi € R 3 , 
then C can be viewed as the 3-D rotational matrix. The main 
idea behind (12) is that the original control input for reaching 
consensus is now rotated by some angles. The closed-loop 
system can be expressed in a vector form, whose stability can 
be determined by studying the distribution of the eigenvalues 
of a certain transfer matrix. Main research work was conducted 
in [170]— [173] to design proper algorithms and analyze the 
collective motions for systems with single-integrator kinemat- 
ics and double-integrator dynamics. Analogously, the collec- 
tive motions for a team of nonlinear self-propelling agents 
were shown to be affected by the coupling strength among the 
agents, the time delay, the noise, and the initial states [174]. 

Note that the collective motions for nonholonomic mobile 
robots were also studied recently in, e.g., [175], [176]. Al- 
though the study in [170]-[173] is different from that in [175], 
[176], similarities exist in the sense that all agents will not 
move to the weighted average of the states of neighboring 
agents, but to some offsetted state. Noticeably, the offsetted 
state in [170]— [173] is properly designed, yet the one in [175], 
[176] is induced by some special nonlinear system dynamics. 

In the study of formation producing with linear closed-loop 
systems, it is observed that the associated system matrix has 
two interesting properties: (1) the existence of at least one 
zero eigenvalue, and (2) the existence of at least one pair of 
eigenvalues on the imaginary axis. The two properties play 
an important role in the formation producing problem under 
a fixed network topology. However, the two properties might 
not be able to solve the formation producing problem under 
a switching network topology, which remains a challenging 
problem due to the complexity in the analysis of switching 
systems. 

2) Lyapunov Function Approach: Although matrix theory 
is a relatively simple approach for stability analysis of the 
formation producing problem, it is not applicable in many for- 
mation producing scenarios, especially with nonlinear systems. 
It is then natural to consider the Lyapunov function approach. 

By using the Lyapunov function approach, several typical 
formation producing scenarios have been studied, including 
the inverse agreement problem [177], leaderless flocking and 



stabilization [178]— [188], and circular formation alike [175], 
[176], [189]— [192]. In the inverse agreement problem [177], 
the objective is to force a team of agents to disperse in 
space. Roughly speaking, for the single-integrator kinemat- 
ics (1), the corresponding control input takes the form of 
Ui(t) = £™=i b ij(\\ x i -Xj\\)[xi(t) - xj(t)], where 
is a nonnegative function. Assuming that each agent can 
communicate with all other agents within a radius R, the 
agents will disperse in space with the relative distance between 
any two agents being larger than R. 

For the case of leaderless flocking, research has been 
conducted to stabilize a group of agents towards some de- 
sired geometric formation, where the inter-agent interaction is 
described directly or indirectly by some nonnegative potential 
function gardless of the final group veloc- 

ity. Some notable properties of Vy(||o;, — Xj||) includes: (i) 
— 5Gj-||) achieves its minimum when \\xi — Xj\\ is equal 
to the desired inter-agent distance between agents i and j, 
(ii) T/y (||.Ti — Xj\\) increases as \\xi — Xj\\ decreases from the 
desired distance to zero and l^j(||xi — Xj\\) could approach 
infinity as \\xi — Xj\\ approaches zero, and (iii) Vij(||a;j — Xj\\) 
increases as \\xi — Xj\\ increases from the desired distance to 
the maximum communication range R. The basic idea behind 
the potential function Vij(||a;i — Xj ||) is to drive the inter- 
agent distance to the desired value while avoiding possible 
inter-agent collision. The corresponding control law for each 
agent is usually chosen to be the same as the corresponding 
consensus algorithm except that the Xi — Xj term is replaced 
by V Xi y(||xs — here. A fundamental limitation is that 
all agents will normally converge to some (constant) inter- 
agent configuration locally in the sense that some nonnegative 
potential function achieves its local minimum. Accordingly, 
the inter-agent distance might not converge to the desired value 
globally. It is an interesting future research topic to study how 
to ensure the desired inter-agent distance be achieved globally 
under a properly designed control algorithm. In addition, the 
network topology associated with a team of agents is usually 
assumed to be undirected, which is not applicable to many 
practical systems which are described by directed networks. 

For the case of circular formation and the like, the main 
research in [175], [176], [189], [191], [193] was devoted to 
the collective motion for nonhonolomic mobile robots with 
dynamics given in (6). Denote r; = x,i + Li/i, where l = 
Then, (6) becomes r, = Uie L6i , Q\ — w,, i = 1,---,N. 
Due to the nature of the nonlinear dynamics, a consensus- 
like algorithm often renders a circular-like ultimate formation 
where the trajectories of all agents are circular and the relative 
phase difference (namely, 8i — 8j) is constant. The current 
work mainly focuses on the case when all agents share 
a common unit speed. Similar circular-like formation was 
analyzed in [190], [192], where the system dynamics are 
different from (6) but share a similar nonlinearity. Due to the 
nonlinearity of the system dynamics, it is a challenging task 
to incorporate time delay, disturbances, quantization, etc, into 
the existing research. 

3) Graph Rigidity: For a network with a given number 
of agents, the edges are closely related to the shape of 
formation. Roughly speaking, if enough information regarding 



10 



edge distances for a team of agents is available, the geometric 
structure of all agents is determined. Then the graph for the 
agents is rigid. According to [194], a graph of n agents is 
rigid if at least 2n — 3 edge distances are available. 

Motivated by the graph rigidity, research has been conducted 
in [ 195]— [ 199] to drive a group of agents to the desired inter- 
agent configuration by ensuring that a certain number of edge 
distances are identical to the desired ones. The graph rigidity 
recovery after loss of an agent has also been investigated. 
Compared with other formation producing algorithms which 
require edge vector information (;' ), less information 

is required in edge distance information (i.e., \\xi — Xj\\). As 
a tradeoff, some unstable equilibria, such as a collinear initial 
configuration (the initial states of all agents are linearly depen- 
dent) and a common initial state (the initial states of all agents 
are identical), rather than the desired inter-agent configuration, 
might exist. In practical applications, it is important to design 
proper control algorithms such that a team of agents can avoid 
converging to the unstable equilibria. 

4) Receding Horizon Approach: Receding horizon con- 
trol (RHC), known also as model predictive control, has 
been introduced into the formation stabilization problem. By 
nature, RHC is a finite-horizon optimization problem. The 
employment of RHC in the formation stabilization problem 
is motivated by the fact that RHC is more capable of dealing 
with constraints. 

The main research on this topic [200]-[202] has been 
devoted to deriving proper distributed control algorithms for 
a team of agents such that they can reach some desired 
formation by optimizing some finite-horizon cost functions 
in scenarios with or without time delay. Because RHC is 
essentially an optimization-based control strategy, the dis- 
tributed control algorithms are typically given by solving 
optimization problems. Therefore, more computational time 
is required by RHC than other control approaches. Therefore, 
the potential computation-induced time delay needs to be taken 
into consideration in practical applications. 

B. Formation Tracking 

Although formation control without a group reference is 
interesting in theory, it is more realistic to study formation 
control in the presence of a group reference because it may 
represent a control objective or a common interest of the whole 
group. This scenario is now reviewed in this subsection. 

1 ) Matrix Theory Approach: Similarly to the case of for- 
mation producing, matrix theory is often used in the study of 
the formation tracking problem. 

An interesting problem in formation tracking is to design a 
distributed control algorithm to drive a team of agents to track 
some desired state. For example, given the single-integrator 
kinematics, control algorithms were designed in [203], [204], 
where the algorithms are similar to those consensus algorithms 
except for that an extra term is introduced here due to the 
existence of the group reference. If properly designed, all 
agents can track the group reference accurately as reported 
in [203], with bounded tracking errors analyzed in [204], 
where a discretized version in [203] was considered. It is 



worth mentioning that the group reference can be arbitrarily 
chosen as long as its derivative is bounded. In [205]-[207], 
the synchronization of a group of linear systems to the output 
of another linear exosystem was investigated with or without 
parameter uncertainties. In [203], [204], a general group 
reference was discussed while in [205]-[207] a general system 
model was considered. How to solve formation tracking for 
general linear systems with a general group reference is still 
an open problem. 

The formation tracking problem can be converted to a tradi- 
tional stability problem by redefining the variables as the errors 
between each agent's state and the group reference. Then, 
the formation tracking problem is solved if the corresponding 
errors can be driven to zero. However, the formation producing 
problem, in general, cannot be solved in this way. Therefore, 
under a switching network topology, the formation tracking 
problem is generally easier to manager than the formation 
producing problem. 

2) Lyapunov Function Approach: Due to the broad applica- 
tions of the Lyapunov function approach in stability analysis, 
it has become an important tool in the study of the formation 
tracking problem as well. 

Flocking with a dynamic group reference has been stud- 
ied recently [208]-[212], where the objective is to design 
distributed control algorithms such that a team of agents 
move cohesively along the group reference. Compared with 
leaderless flocking where no specific final group velocity is 
required, the study of flocking with a dynamic group reference 
is much more challenging both theoretically and technically 
due to the existence of the dynamic group reference and the 
requirement on the cohesive movement of the agents along 
the dynamic group reference. In other words, the agents not 
only have to maintain some desired geometric formation but 
also need to follow the group reference as a whole. The 
combination of the two objectives makes the problem much 
more difficult than the leaderless flocking problem where only 
the first objective is involved. If enough information of the 
group reference is known, such as the acceleration and/or 
velocity information of the group reference, flocking with 
a dynamic group reference can be solved by employing a 
gradient-based control law [208]-[210]. Another approach was 
proposed in [211], where a variable structure-based control 
law was used to solve the problem with less information 
required. Similarly to the study of the leaderless flocking 
problem, the existing approaches on flocking with a dy- 
namic group reference can only reach a local minimization 
of certain potential functions because the potential function 
is generally unspecified but satisfies the conditions stated in 
Subsection IV-A. Accordingly, the inter-agent distance is not 
identical to the desired one. However, the potential-based 
control can be easily designed to guarantee collision avoidance 
and maintain the initial inter-agent communication patterns. 
Nevertheless, it is still an open problem to accomplish the 
task with global inter-agent distance stabilization, collision 
avoidance, and initial communication pattern maintenance. 

Formation control with a group reference was studied in 
both linear systems [213]— [215] and nonlinear systems [216]— 
[224] when the potential function V(||xj — Xj||) is re- 
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placed by some known functions, generally in the form of 
\\xi — Xj — dij\\ 2 , where dij denotes the desired distance be- 
tween agents i and j. Briefly, the nonlinear systems studied in 
this case include nonholonomic mobile robots (see (6)) [218]— 
[222], rigid bodies (see (7)) [223], [224], and linear systems 
with other nonlinear terms [216], [217]. Compared with the 
flocking problem, the problem studied here is relatively easier 
due to the known V(||a:j — Xj ;||). In general, the inter-agent 
distance can be driven to the desired one. As a tradeoff, 
the collision avoidance and initial communication pattern 
maintenance need to be considered separately. 

C. Connectivity Maintenance for Consensus and Formation 
Control 

In both consensus and formation control problems, it is 
often assumed that the network topology satisfies certain 
fundamental conditions, for example, is connected or has a 
directed spanning tree. However, a practical communication 
model is typically distance-based, i.e., two agents can commu- 
nicate with each other if and only if their distance is smaller 
than a certain threshold, called communication range. This is 
particularly true for sensor networks. In order to guarantee 
consensus or formation control be achieved asymptotically, a 
connectivity maintenance mechanism is essential, which has 
been studied recently. 

The main approach to maintaining the connectivity of a 
team of agents is to define some artificial potentials (between 
any pair of agents) in a proper way such that if two agents are 
neighbors initially then they will always communicate with 
each other thereafter [211], [225]-[234]. In general, the artifi- 
cial potential between a pair of agents grows to be sufficiently 
large (could be unbounded) when the distance between them 
increases to be equal to the communication range. For properly 
designed control algorithms, which are usually composed of 
the gradients of the artificial potentials, the total artificial 
potential is nonincreasing. This then indicates that the initial 
communication patterns can be preserved because otherwise 
the total potential will become larger than the initial total 
artificial potential, as soon as some communication pattern 
is broken. Although this approach provides a systematic way 
to guarantee the connectivity, the corresponding control al- 
gorithms might require infinite large control inputs, which is 
not practical. Meanwhile, it is not even necessary to always 
maintain the initial communication patterns in order to guar- 
antee the connectivity. Therefore, how to find a more effective 
way to guarantee connectivity deserves further investigation. 
In contrast to the studies in [211], [225]-[234], the authors 
in [235] investigated an interesting problem where the number 
of initially existing communication patterns plays a role in 
the connectivity maintenance for the consensus problem with 
single-integrator kinematics (1) and control input (2). Roughly 
speaking, if the initial graph is "sufficiently" connected in 
the sense that each agent has at least a certain number 
of neighbors, consensus can be guaranteed to be achieved. 
Note that the result can only be applied to systems with 
single-integrator kinematics therefore further investigation is 
expected for systems with high-order linear dynamics or 
nonlinear dynamics. 



D. Remarks 

Current research on formation control mainly focuses on 
a fixed formation where the inter-agent distance is fixed. 
Considering practical applications, however, it might require 
the formation be adaptive with respect to the events performed 
by the team of agents. In addition, it is important to consider 
constraints, such as input saturation, quantization, and power 
limitation, in the formation control problem. Meanwhile, the 
robustness is another important factor that deserves consider- 
able attention in real applications where noise and disturbances 
exist. 

In terms of connectivity maintenance for consensus and for- 
mation control, research has been devoted to continuous-time 
systems but practical systems are more suitable to be modeled 
in a discrete-time setting, which makes the study of connec- 
tivity maintenance for discrete-time systems more meaningful. 
In general, the connectivity maintenance for discrete-time 
systems is more challenging due to the fundamental limitation 
of the corresponding control input, which is usually piecewise 
constant rather than continuous. 

V. Optimization 

Optimization is an important subject in the studies of control 
systems. The main objective of optimization is to find an 
optimal strategy subject to some given constraints such as 
cost functions. Recently optimization in distributed multi- 
agent coordination has been studied concerning convergence 
speed and some specific cost functions, which are respectively 
reviewed below. 

A. Convergence Speed 

As discussed above, one important problem in consensus is 
the convergence speed, which characterizes how fast consensus 
can be achieved therefore is desirable to optimize. In this 
regard, the convergence speed is the cost function to be 
maximized. 

Consider a group of n agents with dynamics described by 
the single-integrator kinematics (1). Equipped with (2), the 
dynamical system (1) can be written in a matrix form, as 

X(t) = -CXit), (13) 

where X(t) = [xi(i),--- ,x n (t)] T and C is the Laplacian 
matrix. For a network with a fixed topology, £ is a constant 
matrix. 

Motivated by the observation that the smallest nonzero 
eigenvalue of the Laplacian matrix, \2(£), determines the 
worst-case convergence speed [8], research has been con- 
ducted to maximize the convergence speed [153], [236] by 
choosing optimal weights associated with edges. In contrast 
to [153], [236], where the systems are assumed to have single- 
integrator kinematics, optimization of the convergence speed 
for double-integrator dynamics was considered in [237], where 
the convergence speed is defined in a similar way to the A2 (C) 
for the case with single-integrator kinematics. It is worth 
mentioning that optimal convergence for general linear and 
nonlinear systems is still an open problem. 
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Other than A2<X), another commonly used measure for 
the convergence speed is given by (11). The corresponding 
optimization problem is 

max p, (14) 

where p is defined in (11) and U is a set of admissible 
controllers. Existing research in [154], [238] focuses on the 
case when all agents converge to the average of the initial 
states, i.e., X* = [— Y^i=i x i(0)]l- F° r an arbitrary fixed or 
switching network topology, the optimization problem (14) 
is challenging if X* is unknown. But if X* is chosen as 
[— Yli=i x i(f)]l> tnen the problem becomes much easier. 

B. Specific Cost Functions 

In addition to the fastest convergence speed requirement, 
various cost functions are also subject to minimization. 

One interesting problem studied in this setting is distributed 
multi-agent optimization, which is motivated by solving one 
challenge in wireless sensor networks, namely, to estimate 
the environment parameters and/or some functions of inter- 
est, such as temperature and source location [239]. As a 
simple strategy, each sensor node can send its data to an 
existing central location which can then process the data 
if it is sufficiently powered. However, due to the limited 
power resources and communication capabilities, this strategy 
is often not applicable. An alternative approach to achieving 
a similar objective is to estimate the environment parameters 
and/or some functions of interest locally, which requires much 
less communication bandwidth and power. In wireless sensor 
networks [239], the estimation problem is usually modeled 
as a distributed multi-agent optimization problem. Roughly 
speaking, the objective of distributed multi-agent optimization 
is to cooperatively minimize the cost function Yn=l fi( x )i 
where the function fa : W l n- M represents the cost of agent 
i, known by this agent only, and x £ M." 1 is a decision vector. 
In [239], an incremental subgradient approach was used to 
solve the optimization problem for a ring type of network. It 
should be noted that [239] does not provide much discussion 
on the optimization problem under other types of network 
topologies. 

Ref. [240] was probably the first paper studying the dis- 
tributed multi-agent optimization problem under a consensus- 
based framework. The problem considered therein is formu- 
lated so as to 

minimize J27=i /*( x )> 
s.t. x £ E", 

where each fa : R" — > K is assumed to be a convex function. 
Inspired by the average consensus algorithm and the standard 
subgradient method, a consensus-like algorithm was proposed 

as 

n 

Xi(k + 1) = 2J a ij(k)xj(k) - agi(xi(k)), (15) 

where a is the step size and gi{xi(k)) is the subgradient of 
fi(x) at x — Xi(k). In [239], Y^=i a ij(k)xj(k) in (15) was 
replaced by Xi-i(k) with xo(fc) = x n (k — 1). Note that the 



algorithm (15) can only find sub-optimal solutions, determined 
by the constant step size a. Further results on this topic can be 
found in [241]-[243], where a similar distributed multi-agent 
optimization problem was studied within various scenarios, 
under constraints [241], over random networks [242], and 
with broadcast-based communications in an asynchronous 
setting [243]. In the existing literature, time delay and dis- 
turbances have not been taken into consideration. Therefore, 
it is important to consider time delay and disturbances in the 
distributed multi-agent optimization problem due to their wide 
existence in wireless sensor networks. 

In addition to the distributed multi-agent optimization prob- 
lem, where the cost function is a sum of a series of convex 
functions, distributed optimization has also been considered 
for both infinite-horizon cost functions [244]-[248] given by 
J, = f™[X T (t)QX(t) + U T (t)RU(t)}dt and finite-horizon 
cost functions [249]-[252] given by J f = J* f [X T (t)QX(t) + 
U T (t)RU(t)]dt, where X £ W n is the state, U £ R n is the 
control input, and t / is a positive constant. It is worth mention- 
ing that the RHC approach discussed in Section IV-A typically 
has finite-horizon cost functions. Differing from the research 
reported in [240]-[243], which is to find the optimal estimated 
state, the objective here is to find the optimal control laws 
subject to the minimization of certain cost functions. Due to 
requirements of optimizing the cost functions when designing 
the control laws, the computational complexity becomes an 
important problem to study. Meanwhile, the network topology 
plays a significant role in the optimization problem with 
certain cost functions, therefore it is also important to optimize 
the network topology subject to certain cost functions. 

VI. Distributed Task Assignment 

Distributed task assignment refers to the study of task 
assignment of a group of dynamical agents in a distributed 
manner, which can be roughly categorized into coverage 
control, scheduling, and surveillance. Compared with the pre- 
vious studies discussed in Sections III, IV, and V, distributed 
task assignment focuses on the three task-oriented research 
problems, where each topic has its unique features. 

A. Coverage Control 

Coverage control is an active research direction in mobile 
sensor networks, where the objective is to properly assign the 
mobile sensors' motion in order to maximize the detection 
probability. 

Let Q be a convex space with cf> represent the distribution 
density function which indicates the probability that some 
event takes place over Q [253]. Consider a group of n mobile 
sensors whose locations are specified by P = [pi,-- - ,p n }- 
The sensor performance at a point q degrades with respect 
to the distances \\q — Pi\\, which are all described by a 
nondecreasing differentiable function, /. The coverage con- 
trol problem is to find a local controller for each mobile 
sensor such that the following cost function is minimized: 
J = Yh=iS fih-PilDH^dq- This coverage control by 
nature is an optimization problem. Main research on this topic 
was reported in, e.g., [254]-[261], where the coverage control 
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problem was considered in two directions, namely, analysis 
of coverage control under various practical constraints, such 
as limited sensing/communication capacities [254], load bal- 
ancing [260], and nonholonomic mobile robots [261], and 
algorithms for coverage control [257]. Noting that time delay 
and uncertainties have not been considered in the coverage 
control problem, it is interesting to consider the effect of 
time delay and uncertainties in the coverage control problem. 
Moreover, the density function <j> might be time-dependent in 
real systems, which is another interesting research topic for 
further study. 

B. Scheduling 

Another interesting topic in distributed task assignment is 
distributed scheduling, which refers to the scheduling of a 
group of dynamical agents in a distributed manner. Distributed 
scheduling has many potential applications in military and 
civilian sectors, and can be roughly categorized into two 
typical problems, namely sequence optimization [262] and task 
allocation [263]-[267]. The objective of sequence optimization 
is to schedule a team of agents such that some metrics can 
be optimized. For instance, in [262], an optimal scheduling 
sequence was designed to fuel a group of UAVs via dynamic 
programming, where the metric is the total spending time. The 
objective of task allocation is to distribute certain number of 
tasks to a team of agents such that they can balance the total 
tasks. If the number of tasks for each agent is considered a 
variable in the consensus problem, the task allocation might 
be viewed as a consensus problem except that a limitation 
on the total number of tasks for all agents exists. A notable 
feature of the distributed task assignment problem is that 
various constraints may exist due to the physical properties 
associated with the agents. In view of the difference objec- 
tives for sequence optimization and task allocation, it is an 
interesting topic to consider combining both objectives of the 
two problems simultaneously. 

C. Surveillance 

Distributed surveillance means to monitor a certain area by 
using a group of mobile agents in distributed coordination. 
Distributed surveillance has a number of potential applications, 
such as board security guarding, forest fire monitoring, and oil 
spill patrolling. 

The main motivation of distributed surveillance is that a 
team of agents can monitor a given (large) area more effec- 
tively than a single agent when the team of agents works in a 
cooperative fashion. Accordingly, an important research prob- 
lem in distributed surveillance is to design environment-based 
cooperative control laws for all coordinated agents such that 
the given area can be monitored efficiently. Recent research 
on distributed surveillance has been reported in [268]-[275], 
where a number of physical limitations were identified and 
considered such as time delay and uncertainties [271], [274], 
collision avoidance between agents [272], and heterogeneously 
distributed agents [274]. The current research is conducted 
under the assumption that each agent has enough power such 
that any designed control law can be applied. However, due 



to the power constraints, each agent might be subjected to 
constraints such as bounded control input, limited distance to 
travel, and finite accuracy level, etc., therefore it is interesting 
and important to consider these limitations in the distributed 
surveillance problem in the future. 

VII. Estimation 

Due to the absence of global information, used for achieving 
group coordination in many cases, a distributed estimation 
scheme is often needed for estimation. 

The first problem is to design local distributed estimators 
such that some global information can be estimated asymp- 
totically or in finite time. The second problem is to design 
local controllers based on the local estimators such that the 
closed-loop system is stable. The estimation-based distributed 
control is essentially a combination of both centralized control 
and distributed control in such a way that distributed control 
is used in the estimation of some global information and the 
centralized control idea is used for local controllers design. 
The estimation-based distributed control strategy often inherits 
the merits of both centralized control and distributed control. 
However, it is worth emphasizing that a closed-loop system 
with distributed estimators is much more complicated to 
design than one without distributed estimators. 

Main research on this topic has been reported in [161], 
[276]-[281], where the joint estimation and control problem 
was considered subject to disturbances [276], [278], [279] 
or without disturbances [161], [277], [280], [281]. In [161], 
[276]-[281], a joint estimation and control problem is solved 
in the sense that the distributed estimator is used in the design 
of proper control algorithms such that certain global objective 
can be achieved. Without the aid of distributed estimators, 
the control design is very hard and even impossible. As 
can be noticed from [282]-[288], the distributed estimation 
problem has been considered without much discussion on 
specific control problems. In general, the joint estimation and 
control idea has provided an important approach in the study 
of distributed multi-agent coordination where only neighbor- 
based information is not sufficient for the controllers design. 
On the other hand, in real applications properly designed 
distributed estimators might be used to replace some expensive 
sensors. 

In general, it remains a challenging problem to study 
task-oriented coordination control systems where the use of 
distributed estimation is either necessary or an appropriate 
replacement of certain expensive measurement devices, at 
the costs of difficult control system design and complex 
system stability analysis. Moreover, physical limitations such 
as bounded control input, asynchronous communication, and 
information quantization, could potentially reduce the appli- 
cability of the joint estimation and control scheme in various 
distributed multi-agent coordination systems. 

VIII. Discussions 

This article has reviewed some recent research and devel- 
opment in distributed multi-agent coordination. In addition to 
the theoretical results reviewed above, many experiments have 
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also been conducted to validate the theoretical designs and 
analysis, as can be found in [289]-[294], to name just a couple 
of representative reports. Although the existing theoretical 
research and experiments have solved a number of technical 
problems in distributed multi-agent coordination, there are 
still many interesting, important and yet challenging research 
problems deserving further investigation. Some of them are 
briefly summarized as follows: 

• Quantization effects in distributed coordination algo- 
rithms. The current research efforts focus on studying 
distributed coordination problems with control inputs 
and measurements being analog signals with continuous 
values. However, digital signal processing techniques 
require digital inputs and sampled-data measurements. 
Although quantization effects have been studied in several 
coordination problems, the quantization effect in some 
other distributed coordination problems remain unsolved 
and even untouched. 

• Optimization with integrated individual and global cost 
functions. Optimization problem in distributed coordi- 
nation has been studied with various cost functions. In 
real systems, each individual agent has both local and 
global objectives, contributing to an integration of both 
individual and global cost functions. Therefore, optimiz- 
ing a combined objective is more realistic but also more 
challenging. Another interesting problem is to investigate 
how to balance the individual cost functions and the 
global cost function toward a common objective. 

• Intelligent coordination. Intelligent coordination refers to 
the distributed coordination of a team of agents in the 
presence of artificial intelligence, namely, each agent is 
intelligent in the sense that they can choose the best 
possible responses based on its own objective. Intelligent 
coordination has potential applications not only in engi- 
neering and technology but also in economics and social 
studies. Although research problems, such as pursuer- 
invader problem [295]-[298] and the game theory in 
distributed coordination [299]-[304], have been studied 
recently, there are still many open questions especially 
the understanding of group behaviors in the presence 
of agent intelligence. One interesting problem is how 
to interpret the underlying complex networks as well as 
to stabilize and optimize the network in the presence of 
agent intelligence. 

• Competition and cooperation. Today, most research is 
conducted based on local cooperation but not competi- 
tion. This posts an obvious limitation because competition 
not only exists but also plays an positive role in group 
coordination. For example, due to the lack of competition, 
the final states of the traditional consensus algorithms are 
always limited to be within some region in the state space 
determined by the initial agent states. One interesting 
question is how to introduce competition into distributed 
coordination so as to arrive at different desired regions 
and to improve the system performance by rewarding 
different agents with different benefits. 



• Centralization and decentralization. Although decentral- 
ization shows obvious advantages over centralization, 
such as scalability and robustness, decentralization also 
has its own drawbacks. One shortcoming is that, under 
decentralized protocols, some agents cannot predict the 
group behavior based only on the available local infor- 
mation. Consequently, some group behavior cannot be 
controlled. As a sensible example, current economic cri- 
sis actually illustrates some disadvantages of behavioral 
decentralization. One interesting question, therefore, is 
how to balance decentralization and centralization so as 
to further improve the overall systems performance. 
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